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Abstract 

A brief personal perspective is given of issues, questions, formulations, methods, 
some answers and selected extensions posed by the spin glass problem, showing how 
considerations of an apparently insignificant and practically unimportant group of 
metallic alloys stimulated an explosion of new insights and opportunities in the general 
area of complex many-body systems and still is doing so. 

1 Introduction 

What are spin glasses? The answer to this apparently innocuous question has evolved from 
an initially obscure, if interesting, small special class of metallic alloys to ones concerned 
with the globally pervasive issue of the understanding of emergent complex behaviour in 
many-body systems, the development of new mathematical, simulational and conceptual 
tools, new experimental protocols, new algorithms and even a new class of mathematical 
probability problems. In this article I shall review some of this history and try to expose 
some of the key issues, challenges, solutions and opportunities of the topic. 

2 Random magnetic alloys 

The story starts with magnetic aspects of metallic alloys. In the early 1960's there was 
much interest in the solid state physics community in the behaviour of isolated impurities 
in metals, first with the formation of local magnetic moments on magnetic metal impurities 
in non-magnetic hosts (Anderson 1961) and then with the strong coupling of a localized 
moment to the conduction electrons at low temperatures and its consequences for the 
electrical resistivity (Kondo 1964). The later 60's and early 70's saw the emergence of 
interest in the effects of inter-impurity correlations through spin glasses (see e.g. Coles 
1983 and Mezard et al 1987) and the Kondo lattice (Doniach 1977). 
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The appellation "spin glass" is due to Bryan Coles in the late 60's to label the low 
temperature state of a class of substitutional magnetic alloys, typified by CuMn or AuFe, 
with finite concentrations of the magnetic ions Mn or Fe in the non-magnetic hosts Cu 
and Au. The reason for the name is two-fold, first that in the state the magnetic moments 
(traditionally called "spins" ) on the magnetic ions seem to freeze in orientation but with- 
out any periodic ordering (so conceptually reminiscent of the amorphous freezing of the 
locations of atoms in a conventional (structural) glass), and secondly that the low temper- 
ature specific heat is linear in T, again a feature of conventional glasses. Experiments at 
that time indicated a non-sharp onset of the state as the temperature was reduced from 
the paramagnetic one, suggesting a rapid onset of sluggishness but not a phase transition, 
again as believed to be characteristic of glasses. There were attempts to understand the 
behaviour in the 60's but nothing very extensive. 

But then more accurate experiments in the early 1970's exposed a new source of theo- 
retical interest, an apparently sharp phase transition signalled by a cusp in the magnetic 
susceptibility when external magnetic fields were kept very small (Cannella and Mydosh 
1972). This had to be a new type of phase transition and therefore worthy of extra notice. 
But still theoretical work was minimal until Edwards and Anderson (1975) produced a 
paper that at one fell swoop recognised the importance of the combination of frustration 
and quenched disorder as fundamental ingredients, introduced a more convenient model, 
a new and novel method of analysis, new types of order parameters, a new mean field the- 
ory, new approximation techniques and the prediction of a new type of phase transition 
apparently explaining the observed susceptibility cusp. This paper was a watershed. 

Edwards and Anderson's new approach was beautifully minimal, fascinating and at- 
tractive but also their analysis was highly novel and sophisticated, involving radically new 
concepts and methods but also unusual and unproven ansatze, as well as several different 
approaches. So it seemed sensible to look for an exactly soluble model for which their 
techniques could be verified. Such a model was suggested by Sherrington and Kirkpatrick 
(1975). It extends the Edwards- Anderson model, in which exchange interactions are range- 
dependent and effectively short-range, to one with interactions between all spins, chosen 
randomly and independently from an intensive distribution (and so 'infinite-ranged' but 
not uniform). It offered the possibility of exact solution in the thermodynamic limit and 
an exact mean field theory, in analogy but subtle extension of the infinite-range ferromag- 
net for which naive mean field theory is correct. Study of the SK model, or the mean 
field theory of the EA model that it defines, has proven highly non-trivial and instructive, 
and opened many new conceptual doors. It has also proven to be an entry point to many 
applications and extensions, which are still ongoing. 
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2.1 More details 



2.1.1 Experimental spin glasses 

Let me be more explicit. The original experimental spin glasses can be characterised by 
Hamiltonians 

H = ~J2j(R i -R j )S i -S j , (1) 

hi 

where the i, j label magnetic ions with Heisenberg spins Sj and locations Rj and J(R) 
is an exchange interaction which oscillates in sign as a function of the spin separation. 
In metallic systems the origin of J(R) is the Ruderman-Kittel-Kasuya-Yoshida (RKKY) 
interaction. In the original alloys the disorder is substitutional on a lattice. 

2.1.2 Edwards- Anderson 

What Edwards and Anderson (correctly) surmised was that the important aspect of Q 
is the combination of frustration, corresponding to the fact that the spins receive con- 
flicting relative ordering instructions (as a consequence of the oscillation of the exchange 
with separation), and the quenched disorder in the location of the spins. For theoretical 
convenience they effectively replaced the Hamiltonian by one that can be written as 

H = -- J ij S i ' Sj, (2) 

with spins on all the sites of a lattice but the J^- between neighbouring spins and cho- 
sen randomly from a distribution having weight of either sign. 1 They further chose the 
distribution to be Gaussian of zero mean, thereby both eliminating the possibility of any 
conventional order (with spatially uniform or periodic magnetization) and also having a 
minimal one-parameter characterization 2 . This necessitated the introduction of a new 
form of order parameter to describe magnetic freezing without periodicity. In fact EA 
gave two versions: one based explicitly on temporal freezing, 

q= lim q(t,t + r); q(t, t + r) = AT 1 YVSi(t) • Si(t + r)>, (3) 

t— >00,T— >00 ^ ' 

i 

where (•) refers to a dynamical average, and the other based on ensemble- averaging, 

g = A^£|<S,,)| 2 , (4) 

i 

1 In fact the EA Hamiltonian was first written explicitly by Sherrington and Southern (1975). 
Actually, the Gaussian choice for the single parameter description was also useful for the further 
analytic methods employed. The alternative simple single-parameter symmetric distribution having two 
delta functions of equal weight at ± J has often been employed in (later) computer simulations. 
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with (•) now referring to an ensemble- average restricted to one symmetry-breaking macro- 
state. The phase transition is signalled by q becoming non-zero. 

EA did not attempt a full solution but used several new variants of mean field theory, 
all requiring novel treatment beyond those conventional for a simple ferromagnet. The 
most sophisticated of them introduced and employed the so-called 'replica trick' which 
replaces the average of the logarithm of the partition function 3 , the physical generating 
function 4 , by the limiting behaviour of a partition function of an effective periodic system 
of higher dimensional spins: 

hTZ = lim d/dn(W) = lim d/dn{ TT Z(a)) = lim dZ cS (n)/dn, (5) 

a=l,...,n 

where the overbar refers to the average over the distribution of the J, Z is the usual parti- 
tion function Z = Tr/^i exp{— f3H}, Z(a) is the partition function for spins with dummy 
labels a and Z e g(n) is the partition function of a periodic Hamiltonian H c d({af}) of effec- 
tively higher dimensional pseudo-spins with extra replica labels a = 1,. . . ,n and higher 
order interactions now between spins with different replica, as well as site, labels. Within 
this new description EA devised a new mean field theory with a new order parameter 
measuring inter-replica overlap 

<fP = N- iy £ i S? -Sf; a + fi. (6) 

i 

To go further, however, they employed several assumptions and approximations whose va- 
lidity was difficult to assess, although they do yield results with some qualitative similarity 
to several experimental features. 

2.1.3 Sherrington-Kirkpatrick 

In view of the many uncertainties of the EA analysis and the fact that the model was 
surely not soluble with current techniques, it seemed sensible to look for a model in which 
a mean-field theory might be exact. Since the conventional ferromagnet is soluble in 
the thermodynamic limit provided that all spins interact equally with one another and 
correspondingly the exchange interaction scales inversely with the number of spins, it 
seemed reasonable to look for an analogue in the spin glass problem. This led to the 
formulation of the Sherrington-Kirkpatrick (SK) model whose Hamiltonian is similar to 

3 The argument for studying the average of the logarithm of the partition function is that the physical 
quantities it generates should be self-averaging, independent in the thermodynamic limit of the specific 
instance of choice of the disorder. 

4 Certain physical observables can already be expressed as derivatives of In Z with Z defined as above 
with the bare H. Others, in principle, require the addition to H of terms involving appropriate conjugate 
fields so that desired observables follow from derivatives of In Z with respect to these fields. 
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that of EA but with interactions between all spins, chosen randomly and independently 
from a distribution whose mean and variance scale inversely with the number of spins. 
Simplifying to Ising spins and allowing for a ferromagnetic bias and an external field, the 
SK model is characterised by 5 

H = -Y / J ijWj- h Y l <n * = ±1 ; Ji j ^-d.;J~=J /N,4 j = J 2 /N. (7) 

(ij) i 

Despite its apparent simplicity, this model has turned out to expose many subtleties; for 
statistical physics, for mathematical physics and for probability theory; as well as having 
much wider application relevance. Extensions to other related models with extensive and 
super-extensive constraints independently drawn from identical (intensive) distributions 
have led to further novelties and applications. In this introductory perspective I shall 
restrict discussion to outlines at the level of conceptual theoretical statistical physics, 
leaving mathematical rigour to other authors. 



2.1.4 Replica theory 

Within the replica theory of EA but applied to the SK model the averaged free energy 
can be expressed in a form 



F = -Tin Z = —T lim A Tr exp ( / ( V of , V afaf] ) , 

n->0 dn {af;a=l,...,n} \ \^ ^ ) j 



(8) 



in which / involves the spin variables only in the form of sums over all sites and those 
sums only up to quadratic order. Hence, by introducing auxiliary (macroscopic) variables 
to linearize these sums, the trace over the spins may be taken to yield 

~F = —ThiZ = -T lim j JJ dm a JJ dq a/3 exp[-N<S>({m a ; q a(3 })} (9) 
n_> a=l,...,n (a/3) 

with $ intensive. Thus, provided the limit n — > and the thermodynamic limit iV — > oo 
can be inverted, the method of steepest descents in principle yields a solution determined 
by an extremum of & . However, to take the limit n — > an appropriate analytic form 
continuable to non-integer n is needed and the correct way to achieve this is not obvious. 
EA and SK both used the natural 'replica-symmetric' ansatze, 

m a = m, all a; q af3 = q, all a + /3. 6 (10) 



'We use notation (ij) to denote a pair of unequal sites. 
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This ansatz also yields the identifications 



m = (o-j) and q = \{<Ti)\' 



(11) 



Already it gives many features qualitatively similar to ones found experimentally. In fact, 
though, it does not in general give a stable solution (de Almeida & Thouless 1978) and 
a much more subtle replica-symmetry-breaking ansatz for q is needed to yield stability in 
all regions of control-parameter space. The Parisi ansatz (Parisi 1980) has satisfied this 
need and passed all subsequent stability tests. 

Let me first describe Parisi's ansatz in terms of its original replica theory formulation 
and only turn later to its physical interpretation. q a @ can be viewed as an n x n matrix 
with zeros on its diagonal elements 7 . The Parisi ansatz may be viewed as the result of a 
sequence of operations in which (i) n(= mo) is initially considered as an integer which is 
subdivided sequentially into an integral number of smaller intervals; first into n/mi blocks 
of size mi, then each of the mi blocks into mi/m2 blocks of size mi and so on sequentially, 
with all the m; integers and the ratios mj/mj+i also integers, until m^+i = 1 (ii) q a @ is 
taken to have the value qi if I (a /mi) = I(P/m,i), /(a/m,j + i) ^ /(/3/m,j + i) where I(x) 
is equal to the smallest integer greater than or equal to x, as illustrated below for the 
sequence n = mo = 12; mi = 4; m-2 = 2; m,3 = 1 
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(iii) in the limit n — > the m are continued to real values with < mi < mi < ■ ■ ■ < 
rrik < m k+i = 1 and q is replaced by a function q{x) given by q(x) = qf, mi < x < mi + i 
(i = 1, ...,k) with x in [0,1], (iv) the limit k — > oo is taken. Insertion into eqn. JHJ) 
yields a functional integral which in the limit N — > oo is extremally dominated and yields 



6 One also requires that the extremum of <E> be a minimum for the single-replica order parameter m a 
but a maximum for the two-replica order parameter q a/3 . 



7 Some authors take q° 
so taken explicitly. 



as unity (c.f. an extension of eqn. ijfjj) but here I am assuming the aa term is 



6 



self-consistency equations for the dominating q(x); hereafter q(x) is taken to refer to this 
extremal function, which is the the mean-field order function for the problem. For different 
regions of the (J, Jo, h, T) parameter space the stable solutions are of one of two forms: 

(i) q(x) = q = constant; replica-symmetric (RS) 

(ii) q(x) = qo for < x < x±, monotonically increasing smoothly between x\ and X2, 
and q(x) = q\ for X2 < x < 1; full replica symmetry breaking (FRSB) 8 . 

They are separated by a plane in (J, Jo, h, T) which marks a continuous transition, with 
RSB on the higher-J side. 

Replica symmetry breaking signals the existence of many non-equivalent macrostates. 
q(x) provides a measure of the extent of similarity between these states. It follows 
from consideration of the concept of overlaps (Parisi 1983). The overlap between two 
macrostates S, S' is defined by q ss = N^ 1 Yli( a i) S ( a i) S > where (-} s refers to a thermo- 
dynamic average over macrostate S, and the distribution of overlaps is given by P(q) = 
Yls S' WsWs'S(q — q ss ) where Ws is the probabilistic weight of state S, given in equi- 
librium by Ws = ex.p(—f3Fs)/J2s' ex P( — P^S') where F$ is the free energy of macrostate 
S. The relation to q(x) is P(q) = J dx5(q — q(x)) = dx/dq. Consequently it follows that 
an RS system has a single macrostate (aside from trivial global inversion or rotation), 
whereas FRSB implies a hierarchy of non-equivalent relevant macrostates at the temper- 
ature of interest 9 . This in turn implies that the macroscopic dynamics will be slow and 
glassy and that practical equilibration will be very difficult to achieve. Already, however, 
the existence of RSB predicts different kinds of response functions; for the susceptibility 
one may experience either single-macrostate response xss = Xea = T _1 (l — <?(1)) or the 
full Gibbs average \G = J _1 (l — / dxq(x)). These in turn can be identified with the 
experimental zero-field-cooled and field-cooled susceptibilities and used to explain their 
difference in the spin glass phase (see e.g. Nagata et al. 1979); this non-ergodicity was 
already observed before EA in the difference between thermoremanent and isothermal re- 
manent magnetisations (e.g. Tholence and Tournier 1974). The Parisi replica analysis 
also demonstrates a number of other interesting properties (Mezard et al. 1984), such as 

8 For the intermediate approximations mentioned above one would have a fc-step replica symmetry- 
breaking with k + 1 sections of constant q(x) separated by k discontinuities, but it is believed that the 
only stable situations for the SK model are k = (RS) and k = 00 (FRSB). There are stable 1 step RSB 
solutions for several other problems (see section 5). 

9 There are even more macrostates of relevance at different temperatures. 
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ultrametricity (Mezard and Virasoro 1985) 10 and non-self-averaging of certain non-trivial 
overlap measures, but these will not be dwelt upon further here 11 . 

2.1.5 Short-range spin glasses 

'Real' experimental spin glasses have short-range or spatially decaying exchange interac- 
tions, whereas the replica theory above is exact only for infinite-range problems. Many of 
the predictions of mean field theory are mimicked qualitatively in the experiments; some 
are thought to be real, but others are still subjects of controversy in true Gibbs equilib- 
rium although often apparent as non-equilibrium experimental features. The Edwards- 
Anderson model with nearest neighbour interactions is considered representative of such 
real spin glasses but remains without full exact solution. 

2.1.6 Spin glasses on dilute random networks 

A class of model spin glasses with finite inter-spin connectivities, as is the case for EA, 
but range-free and offering the possibility of exact solution, was introduced by Viana & 
Bray (1985) and characterised by an analogue of SK with 

H = — ^^CiCjJijaiUj; random quenched c and J^-j a = 0, 1; Jjj = Jo, Jfj = J 2 , (12) 

(ij) 

where the annealled spins a are located on the quenched vertices of a finite-connectivity 
Erdos-Renyi 12 graph, with a = 1 denoting a vertex, but without the need for inverse N- 
scaling of the exchange distribution. 13 . This problem, which is often considered a 'half- 
way house' between SK and EA, requires more order parameters m a , q a/3 , q a ^ , q a ^ s , . . . 
and, although soluble in RS approximation via a mapping q a P— r = J P(/i){tanh(/?/i)} r , 
also poses greater challenges than SK for FRSB (see e.g. Wong and Sherrington 1988). 

10 This corresponds to the hierarchical clustering of overlaps as illustrated by the branching cartoon 




For a recent review of the topic of overlaps and their interpretation see (Parisi 2004). 
12 In an Erdos-Renyi graph of degree p any vertex is connected to any other with a probability p/N. 
13 A simple extension utilises as underlying network a random graph with fixed degree at each vertex 
(Banavar et. al. 1987). 
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2.1.7 Itinerant spin glasses 

Thus far we have discussed only systems with magnetic moments even in the absence 
of interaction. However, it is well known that ferromagnetism in periodic systems can 
occur not only through the orientation of effectively pre-existing localized moments, as 
typified by Curie- Weiss mean field theory and found in insulating magnets and in some rare 
earth metals, but also through the spontaneous cooperative ordering of metallic itinerant 
electrons, as in Stoner-Wohlfarth ferromagnetism in transition metals. Similarly, one can 
readily envisage itinerant spin glass behaviour (Sherrington and Mihill 1974) and indeed 
it is found in alloys such as RhCo (Coles et.al. 1974). A simple model is given by the 
Hamiltonian 



where the ctj, aj~ are Wannier electron creation and annihilation operators, rii a = a^a^ are 
number operators, and the parameters tij, Vi, Ui depend upon the types of atom at sites 
The simplest instance takes randomly quenched alloys with two atomic types (A, B) 
but with the Uj independent of the atom types and considers only magnetic fluctuations 



in which the Ui take one value U a at the sites associated with atom type A and take 
the another value Ub at the sites associated with atom type B. Of particular interest is 
the itinerant case in which (i) A is not spontaneously magnetically ordered, i.e. (1 — 
UaX°{q)) > 0) where x°(q) is the wave- vector dependent susceptibility associated with 
the bare band structure, (ii) the pure B system is spontaneously itinerantly ferromagnetic 
(so (1 — UbX°(Q)) < 0)) but also (iii) there is no magnetic moment associated with an 
isolated B atom in an A matrix, even in the mean field sense of Anderson (1961). Analogy 
with the phenomenon of Anderson localization (Anderson 1958) leads to the expectation 
of statistical fluctuation nucleation of cluster moments within the conceptual framework 
of Anderson local moment formation, while further cluster interaction can stabilise cluster 
glass behaviour beyond a critical B concentration (Sherrington and Mihill 1974), as well as 
ferromagnetism at a higher concentration. However, in fact isolated paramagnetic cluster 
moments are not necessary precursors for spontaneous spin glass order (as emphasised 
by Hertz 1979), as neither there are local moments in pure itinerant ferromagnets nor 
well-defined bosons in BCS superconductivity above their respective onset temperatures. 

A classical mean field theory follows from (14) by formulating the partition func- 
tion as a functional integral over a Grassmann representation of the electron field (Sher- 
rington 1971), linearizing the interaction term involving the U over auxiliary Hubbard- 
Stratonovich fields, integrating out the electron fields and taking the static approximation. 




(13) 




(14) 
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This yields an effective classical field theory with 

Z = j L>mexp(-/?F(m)), (15) 

where 

F ( m ) = /2 u i m i ~ /2 ' •' /'"'"' A' 1 / ~ /2 UiUjUkUirnimjrnkrrikAijM + (16) 

i ij ijkl 

where rrii =< n^—riii > and xfj > ^ijkv ■■■ are two-, four- and ..-point correlation functions 
of the bare band structure (in real space). Taking the extremum yields a set of self- 
consistent mean field equations which are the analogue of the Thouless- Anderson-Palmer 
(1977) (TAP) equations for the SK equation. The analogy with Anderson localization 
follows from writing these equations as 

U^M % - £ X%M 3 - £ ^ijkiMjM k M[ + .. = 0; Mj = Urn (17) 

j jkl 

and comparing with the Anderson wave-function localization equation 

eifa + ^Ujfa -Efa = 0, (18) 

3 

with disorder in the e»(= Ui^ 1 ); naively, local moment clusters of (17) are related to 
negative energy states of (18) and long range magnetic order is related to the mobility 
edge. But in fact there are more subtle effects, both bootstrap effects as mentioned 
earlier (contained in the non-linear terms of (17)) and effects differentiating spin glass and 
ferromagnetic cooperative order. 

A simple conceptual model of itinerant spin glass ordering, further simplified in the 
EA spirit, is given by an effective field theory with 

Z= / > n^exp(-F(^)); J P(</.) =r^& 2 + «^& 4 -J^ Jijfafau > 0, (19) 
i i i (ij) 

with the Jij random as in EA or SK; this model encompasses local moment spin glasses 
for r < and itinerant spin glasses for r > 0. 

2.1.8 Other induced moment models 

There are other classical models allowing the bootstrapping of magnetic order. One such 
is the spin glass analogue of the induction of ferromagnetism due to exchange interaction 
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lifting of singlet ground state preference of isolated atoms. A simple example is the spin-1 
Ising model 

H = -DY J S 2 i -\Yl J ^ S ^ S = 0,±1. (20) 

i ij 

If D > then the system behaves analogously to the usual spin 1/2 Ising model, but if 
D < then in the absence of J the ground state preference is for non-magnetic Si = 0. 
However even if D < a sufficient exchange can self-consistently lift the preference to the 
magnetically ordered state via a first-order transition. If the Jij are quenched random as in 
the SK model, this system is known as the Ghatak-Sherrington (GS) model (1977) and has 
induced spin glass behaviour; it has been analysed extensively in FRSB by Crisanti and 
Leuzzi (2002). The Fermionic Ising Spin Glass (FISG) model (Rosenow and Oppermann 
1996) is closely related (Perez Castillo and Sherrington 2005). 

2.1.9 Vector spin glasses 

Magnetic alloy spin glasses are not restricted to Ising systems. Indeed Heisenberg magnets 
are more common experimentally. It is straightforward to extend the exactly soluble mod- 
els to encompass vector spins (see e.g. Sherrington 1983). In the absence of a magnetic field 
or a ferromagnetic component there is little change of note beyond the extension of ran- 
dom spin glass freezing to the full spin dimensionality 14 . Within the infinite-range/ mean 
field model, an axial symmetry-breaking due to an applied field or to ferromagnetism still 
permits a spin glass freezing in the orthogonal directions (Gabay and Toulouse 1981) with 
strong onset of transverse non-ergodicity and induced weaker longitudinal non-ergodicity, 
crossing over to strong RSB in all directions at a lower temperature (Cragg et a. 1982, 
Elderfield and Sherrington 1982, 1984). Anisotropy effects can also be included (Cragg 
and Sherrington 1982b). 

3 Discontinuous transitions 

For the case of conventional 2-spin interactions, as employed in both the SK and EA 
models and believed to be appropriate for conventional experimental magnetic alloy spin 
glasses, mean field theory yields full replica symmetry breaking once the spin glass state 
occurs 15 . However, in extensions which lack reflection and definiteness symmetries, such 
as p-spin models for p > 2 (Crisanti and Sommers 1992) or Potts or quadrupolar spin 

14 But it might be noted that the choice of Ising spins by SK led to the 'smoking gun' of negative entropy 
at T=0 and the realisation that there was a subtlety, which eventually led to Parisi's ansatz and beyond; 
negative entropy at T=0 was a known pathology of continuous classical spins but should not occur for 
discrete spins such as Ising. 

15 Although it remains controversial as to whether any RSB holds in short-range systems 
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glasses beyond critical Potts or vector dimensions (Gross et al 1985, Goldbart and Sher- 
rington 1985) one finds that the spin glass transition is discontinuous to one step of replica 
symmetry breaking with finite overlap magnitude (D1RSB) 16 . This behaviour is thought 
to be characteristic also of (even short-range interaction) structural glasses, in which crys- 
tallization is dynamically avoided in favour of self-consistent glassiness. 

4 Beyond magnetic alloys 
4.1 Complex many body problems 

The formalism and concepts developed for model magnetic alloys have found significant 
application more generally; in particular for a large class of problems that can be charac- 
terised by control functions of the form 



where the i, j are microscopic identification labels; the {Jij...k} symbolise a set of quenched 
parameters depending on one or more of the identification labels and in general different for 
different labels; the {<%...;} symbolise the (annealled) microscopic variables again depend- 
ing on one or more identification variables; and the {X} are macroscopic intensive control 
variables. The specific identifications of the {J,S,X} can however be quite different, as 
also the manner of operation of the control function. In the spirit of statistical physics 
and probability theory one often concerns oneself with problems in which the {J, X} are 
drawn from intensive distributions independent of the specific labels. 

4.1.1 Examples 

We have already seen one example in the case of a magnet with the i labelling the spins, the 
J exchange interactions, the S spin orientations, X the temperature and H the Hamil- 
tonian determining the distribution of the S through the Boltzmann measure. Other 
examples include: 

(i) The Hopfield neural network: Here the i label neurons, {Si} indicate the states of 
the neurons as firing or not firing, {Jij} label synaptic efficacies given in terms of 
(randomly chosen quenched) stored patterns \i = 1, . . . ,p = aN by = 

N^ 1 ^i^j: X = T = /3 _1 is a measure of the rounding of the sigmoidal response 

16 In a Potts or quadrupolar model for a range of intermediate Potts or vector dimensions the transition 
to 1RSB is continuous (Elderfield and Sherrington 1983, Goldbart and Sherrington 1985, Sherrington 
1986); a similar transition to C1RSB occurs in a p > 2-spin model in a sufficient applied field (Crisanti 
and Sommers 1992). Except for spherical spins, there is also a lower temperature transition from 1RSB to 
FRSB (Gardner 1985, Gillin et al (2001). 



H = H({Jij_k}, {Sij...i}, {^}), 
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of a neuron to the sum of its incoming signals, H = — \ Ylij JijSiSj and PdS"}) ~ 
exp(—(3H) characterises the stationary macro firing states. From the neural retrieval 
perspective, however, interest is not in the full Gibbs average but rather in the 
individual retrieval macrostates with macroscopic overlaps = N^ 1 J2i^i '{Si) 
with the patterns coded in the {</}; retrieval corresponds to a finite overlap with 
a single pattern and is an analogue of ferromagnetism in the examples of section 
2. Spin glass states do occur due to pattern interference but are not the desired 
states in neural operation and their dominance indicates breakdown of retrievable 
memory 17 . 

(ii) Hard optimization: Here the objective is to minimise a cost function H as a function 
of variables {S} with constraints {</}. An example is the problem of partitioning 
the vertices i of a random graph into two groups of equal size but with the minimum 
number of edges of the graph between the two groups. This can be formulated as 
finding the ground state of a Viana-Bray-like spin glass. Consequently it can be 
studied by an analogue of the procedure of studying the thermodynamics of the 
VB spin glass. If the interest is in finding the average minimum spanning cut then 
replica procedure may be employed, inventing an artificial annealing temperature T 
and taking it to zero at the end of the calculation. Of course the actual calculation 
involves all the subtleties of replica symmetry breaking and computer simulation 
involves all the corresponding issues of slow glassy dynamics 18 . Another optimization 
problem in artificial neural network theory is to determine the maximum number of 
patterns which can be stored and retrieved with a specified maximum error; in this 
case the variables are the synaptic efficacies and the quenched parameters are the 
stored patterns. More recently many other optimization problems have been studied 
by techniques derived from spin glass studies. 

(iii) Error- correcting codes: One procedure for coding and retrieval is to code the in- 
formation to be transmitted in the form of exchange interactions whose insertion 
into an effective magnetic Hamiltonian yields a ground state which identifies the 
desired message. In practice, however, transmission lines add noise and retrieval 
is required to best eliminate the effects of the noise. This yields yet another opti- 
mization problem, with best retrieval resulting from the introduction of an effective 
retrieval temperature-noise matching that on the line 19 . Indeed there are several 
other problems in which the optimal character of noise matching can be demon- 
strated. 

17 For further discussion see for example Sherrington (1992) or Nishimori (2001). 

18 Simulation also exhibits the spin-glass features of ultrametricity and non- self- averaging (Banavar et al 
1987). 

19 Again see Nishimori (2001) for further details. 
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5 Dynamics 



Thus far discussion has been about equilibrium or quasi-equilibrium. However, often 
one wishes to consider dynamics, including away from equilibrium; 20 , indeed if detailed 
balance is not present one cannot use usual Boltzmann equilibrium theory. As before, we 
are normally interested in systems characterised by simple distributions. Again one can 
utilise the general picture of a controlling function as in (21) but now operating in an 
appropriate microscopic dynamics (and without necessarily symmetries such as Jjj = Jji). 
The analogue of the use of the partition function for thermodynamics is to use a dynamical 
generating functional (de Dominicis 1978) which can be expressed symbolically either, for 
random sequential updates, as 

Z(A) = J DS(t)US (eqn of motion) exp(J dtA(t) ■ S(t)\ , (22) 

where the integral is over all variable paths in the full space-time, the 115 term indicates 
that the microscopic equations of motion are always satisfied and the A • S term is a 
generating term, or, for parallel updates, as 

Z{A) = JudS{t)]JW(S{t + l)\S(t))P {S)exp(^2A(t)-S(t)y (23) 

where Pt(S) indicates the ensemble distribution of S at time t and W(S(t + l)|S(t)) 
indicates the probability of updating from S(i) to S(t + 1). With suitable Jacobian nor- 
malization (not shown explicitly) Z(A = 0) = 1 and one can average over the quenched 
disorder without need for replicas; instead of interactions between replicas one gets effec- 
tive interactions between different epochs. In the case of range-free problems one can again 
eliminate microscopic variables in place of macroscopic ones by the artifice of introducing 
new variables via relations such as 



= f dC(t,t')S(C(t,t') - N- 1 Y J S l (t)S i (t')) 

= J dC(t,t')dC{t,t f ) exp^iC(t, t')(C(t,t') - N^Y^SitySitf)) 



(24) 



and similarly for response functions (involving also operators corresponding to d/dSi{t)). 
One can then integrate out the microscopic variables to leave purely macroscopic measures; 
in the simplest cases of the form 

Z cS ~ J DC {t, t>, t", ...) exp(iV$({C})), (25) 



20 Note that whereas in real physical situations the true microscopic dynamics is determined by nature, in 
computer simulations the dynamics is chosen by the simulator and there exists the opportunity to optimise 
that choice. Similarly, the control fields X are choosable. 
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where C is used to denote the generic set and the temporal dependence is two-time for 
full connectivity of the SK type but includes all numbers of different times for VB finite 
connectivity. Steepest descents then yields self-consistent coupled equations for the macro- 
scopic correlation and response functions, although of course boundary conditions need 
care. This is the dynamical analogue of replica thermodynamics. In general, however, 
it is more difficult than replica theory and fewer cases have been solved fully. Also, in 
some cases the convenience of a final expression purely in terms of coupled correlation and 
response functions is not available, although alternative descriptions in terms of ensembles 
of effective single agents can often be obtained. 

An alternative procedure invoking an infinite multiplicity of single-time order param- 
eters has also been considered but will not be pursued here (see e.g. Coolen et al. 1996) 

5.1 Examples 

5.1.1 p-spin spherical spin glass 

One example of the above procedure has been the analysis of the (infinite-range) p(> 2)- 
spin spherical spin glass, of Hamiltonian 

ii<i2<---<i P i 

and obeying Langevin dynamics, for which closed equations in terms of correlation and 
response functions have been obtained (Cugliandolo & Kurchan 1993). In general these 
equations are not restricted to stationarity. Analysis has indicated that above a critical 
temperature, known as the dynamical transition temperature, stationary solutions do 
exist, with C(t,t') = C{t — t') and satisfying the normal fluctuation-dissipation theorem 
and mode-coupling theory, but below this temperature equilibration does not occur, the 
normal fluctuation-dissipation theorem —dR/dC = (3 (where R is the integrated response, 
C is the correlation function and (3 is the inverse temperature) is replaced by a modified 
relation —dR/dC = f3X(C) where X(C) = x(C) with x(q) the inverse of the Parisi 
function q(x), the R and C are now two-time (and non-stationary) and the C-dependence 
of X(C) is instantaneous-parenthetic 21 22 . 

These and related dynamical studies vindicate and quantify the concepts of glassy 
dynamics deduced from the thermodynamic existence of many non-equivalent metastable 
macrostates and the barriers between them. 

21 See for example Parisi (2004) 

22 In this case the onset of RSB is discontunuous and the transition temperature is given differently by 
simple extremization of replica theory and dynamically. The correct comparison with dynamics within 
replica theory is determined using marginal stability. 
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5.1.2 Dynamical SK-model 

In the p(> 2)-spin spherical spin glass model there is only one step of RSB in the replica 
equilibrium theory and similarly only 2 straight slope regions for X(C). The p = 2 SK 
Ising system is more complicated with more structure, corresponding to the hierarchy 
of FRSB, and dynamical analogues of ultrametricity (Cugliandolo and Kurchan 1994). 
Other models can show regions of 1-RSB and of FRSB thermodynamics 23 while it seems 
likely that dynamical vestiges of FRSB may occur in many systems, even with 1RSB 
thermodynamics. 

5.1.3 Minority game 

A rather different example is found in the so-called Minority Game in econophysics (see 
e.g. Challet et al. 2004, Coolen 2004), which mimics a system of speculative agents in 
a model market trying to gain by minority action. In the batch version of this game the 
system obeys microdynamics 

Pi(t + 1) = Piit) -fH Jij sgn Pj (t); hi = N- 1 ^ ii ■ Wj-; Jij = N' 1 ^ • g, (27) 

3 3 

with the i labelling agents, the p unbounded variables corresponding to strategy point- 
weightings, and the £ and Co quenched random vectors in a D-dimensional strategy space. 
This system is soluble along the lines outlined above, utilizing large-iV steepest descent 
domination, in terms of an ensemble of independent agents obeying non-Markovian stochas- 
tic dynamics with ensemble-self-consistently determined coloured noise. On the macro- 
scopic level it exhibits an ergodic-nonergodic transition at a critical value d c of d = D/N, 
asymptotically independent of preparation for d > d c but preparation-dependent for 
d < d c . 

6 Conclusion 

The spin glass problem has yielded many new concepts and techniques in both theoreti- 
cal and experimental physics. These concepts and techniques have in turn inspired new 
insights and practical opportunities in the wider field of complex many-body problems, 
ranging through physics, computer science, biology and economics, with pastures still 
open in these and the social sciences. Most of this work has been on simple models with 
a single level of microscopic timescale (but many resulting macro timescales) but some 
work has started and much remains to do when different parameters are allowed differ- 
ent microdynamic time-scales (as for example in neural networks where both neurons and 

For example the p > 2-spin Ising model has 1-RSB thermodynamic behaviour above a critical temper- 
ature but then FRSB behaviour below; see e.g. Gardner (1985), Gillin et.al. (2001). 
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synapses evolve, the former much faster than the latter, or in biological evolution where the 
timescales of organism operation and species evolution and mutation are very different). 
Although physical systems normally obey detailed balance, others need not (e.g. biological 
or economic or social systems). Most of the theoretical work has been performed at a level 
of uncertain if physically reasonable approximation or assumption. Greater mathematical 
physics rigour is now needed and will be the topic of other authors in this volume. The 
spin glass models have introduced also new concepts in probability theory that are stimu- 
lating new mathematics. Spin glass dynamics poses challenges yet to be investigated with 
mathematical rigour. Much has been achieved but much remains to do. 
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